Approximating Full Conformal Prediction at Scale via Influence Functions

Conformal prediction (CP) is a wrapper around traditional machine learning models, giving coverage guarantees under the sole assumption of exchangeability; in classification problems, CP that error rate at most chosen significance level, irrespective whether underlying model misspecified. However, prohibitive computational costs full led researchers to design scalable alternatives, which alas do not attain same or statistical power CP. In this paper, we use influence functions efficiently approximate We prove our method consistent approximation CP, and empirically show becomes smaller as training set increases; e.g., for 1,000 points two methods output p-values are